Skip to content

Playground: snap hover bboxes to the PDF text layer + show partial extractions - #5

Merged
RATCHAW merged 1 commit into
mainfrom
RATCHAW/fix-hover-bbox-accuracy
Jul 12, 2026
Merged

Playground: snap hover bboxes to the PDF text layer + show partial extractions#5
RATCHAW merged 1 commit into
mainfrom
RATCHAW/fix-hover-bbox-accuracy

Conversation

@RATCHAW

@RATCHAW RATCHAW commented Jul 12, 2026

Copy link
Copy Markdown
Owner

What

Improves the playground's killer interaction — hover a JSON field → highlight its source region — and makes failed runs more useful.

Hover bbox accuracy (snap.ts)

Rescues hover highlights whose model-reported provenance is missing or wrong by locating each field's value in the PDF text layer:

  • Builds visual lines from positioned text runs (reading order, gap-aware spacing).
  • Matches the value's textual forms, including formatted currency (6000$6,000.00), case-insensitively, rejecting matches glued to other word characters.
  • Snaps to the occurrence nearest the model bbox, searching across pages to fix null/out-of-range page indices (e.g. 1-based reporting).
  • Falls back to the model's own bbox when the text can't be located. Images (no text layer) keep drawing the model bbox directly.

Partial extractions on error

On a MissingRequiredFieldsError, the server now carries the extracted-so-far data/fields/usage on the ApiError, and the client renders it under the error banner instead of showing nothing.

Misc

  • Add Gemini 3.5 Flash to the playground model catalog.
  • Add the demo GIF to the README.

Tests

New snap.test.ts covers candidate string generation, snapping, cross-page location, and rescue cases. app.test.ts covers the partial-extraction error path.

🤖 Generated with Claude Code

…tractions

Rescues hover highlights whose model-reported provenance is missing or wrong
by locating each field's value in the PDF text layer (new snap.ts): builds
visual lines from positioned text runs, matches the value's textual forms
(incl. formatted currency), and snaps to the nearest occurrence, searching
across pages to fix null/out-of-range page indices. Images, which have no
text layer, keep drawing the model's own bbox.

Surface partial results on a MissingRequiredFieldsError: the server carries
the extracted-so-far data/fields/usage on the ApiError, and the client renders
them under the error banner. Add Gemini 3.5 Flash to the playground catalog
and the demo GIF to the README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@RATCHAW
RATCHAW merged commit 074ed6a into main Jul 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant